Starburst
Summary
This document covers the information to gather from Starburst in order to configure a Qarbine data service. You can define multiple data services that access the same Starburst endpoint though with varying credentials. Once a data service is defined, you can manage which Qarbine principals have access to it and its associated data. A Qarbine administrator has visibility to all data services.
Starburst Configuration
For Qarbine to interact with Starburst it requires the following information:
- network accessibility,
- Starburst endpoint URL,
- user name and
- password.
Log into your Starburst account.
Using the navigation bar, navigate to your clusters.
Determine the cluster of interest. Each has its own endpoint URL.
Click on the matching
Click the highlighted down arrow as shown below.
Select the cluster containing the data of interest.
Copy the host endpoint URL to a temporary location.
Copy the user to a temporary location.
It is highly recommended that the user account have just read-only permissions. You must also have a password to go along with the given user account. When creating a catalog consider the setting shown below.
Also verify the Qarbine host has access to your Starburst endpoint. This can be done in the Starburst console option shown below.
Qarbine Configuration
Compute Node Preparation
Determine which compute node service endpoint you want to run this data access from. That URL will go into the Data Service’s Compute URL field. Its form is “https://domain:port/dispatch”. A sample is shown below.
The port number corresponds to a named service endpoint configured on the given target host. For example, the primary compute node usually is set to have a ‘main’ service. That service’s configuration is defined in the ˜./qarbine.service/config/service.main.json file. Inside that file the following driver entry is required
"drivers" :[
. . .
"./driver/trinoDriver.js"
]
The relevant configuration file name for non primary (main) Qarbine compute nodes is service.NAME.json. Remember to have well formed JSON syntax or a startup error is likely to occur. If you end up adding that entry then restart the service via the general command line syntax
pm2 restart <service>
For example,
pm2 restart main
or simply
pm2 restart all
Data Service Definition
Open the Administration Tool.
Navigate to the Data Services tab.
A data service defines on what compute node a query will run by default along with the means to reach to target data. The latter includes which native driver to use along with settings corresponding to that driver. Multiple Data Sources can reference a single Data Service. The details of any one Data Service are thus maintained in one spot and not spread out all over the place in each Data Source. The latter is a maintenance and support nightmare.
To begin adding a data service click
On the right hand side enter a name and optionally a description.
Set the Compute URL field based on the identified compute node above. Its form is “https://domain:port/dispatch”. A sample is shown below.
Next, choose the “Trino” driver.
Set the server template to the host endpoint URL as shown below. Remember to add any ‘https://’ prefix.
The server options provide your user, password and catalog information as shown below.
Optionally specify a schema in the Qarbine generalized “database” field as shown below.
Enable the data service for the desired general user group.
Test your settings by clicking on the toolbar image highlighted below.
The result should be similar to the following.
Save the Data Service by clicking on the image highlighted below.
The data service will be known at the next log on time. Next, see the Starburst query interaction and any tutorial for information on interacting with Starburst from Qarbine.
Sample Data Sets
Starburst Provided
Starburst provides sample data sets which are described at https://docs.starburst.io/starburst-galaxy/working-with-data/create-catalogs/sample-data-sets/sample.html#sample-dataset
and
Their content is referenced in the other Qarbine Starburst guides such as the querying one.